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Image processing method of following the deformation of an organ which is deformable over 
time 



The invention relates to a method of processing images belonging to a 
sequence of at least two images having a surface representing an organ or a part of an organ 
which is deformable over time and referred to as the organ surface, said surface including 
characteristic points, denoted marked points, which correspond to each other from one image 
5 to another in the sequence. This invention also relates to an image processing apparatus for 
implementing the method described above. 

The invention finds its application in the field of medical image processing. 
Q The method is in particular applicable to organ images marked by magnetic resonance spatial 

jjj modulation. This marking is visible on the images in the form of marking lines with points of 

W 10 intersection. The marking lines deform following the deformation of the organ. Said 

Llj 

||1 intersection points are then chosen as marked points since the matches between these points 

from one image to another are easily detected. 
P An organ image processing method marked by magnetic modulation with a 

|U view to quantifying the deformation of the organ is already known from the state of the art 

\i 1 5 through the publication by Matthias Stuber et al. entitled "Quantification of the local 
111 heartwall motion by magnetic resonance myocardial tagging". In this document, the points of 

intersection between the marking lines are determined by approximation algorithms based on 
a calculation of potential ("snakes"). In addition, this method uses a mean of the angles with 
respect to the center of gravity in order to quantify the rotation and contraction. 
20 This method has drawbacks. First of all, the approximation algorithms based 

on a calculation of potential mentioned above do not allow a precise determination of the 
positions of the marking lines. Next, the method used by Stuber et al. following the 
determination of the marking lines is concerned only with a calculation of the path of the 
points step by step from paths of the marked points without having a global approximation of 
25 the deformation of the organ. The method used utilizes a mean of the angles with respect to 
the center of gravity for quantifying the rotation and contraction. The result is imprecise since 
it is subject to local errors in determining the marking lines and mathematically incorrect. In 
addition, this method cannot be automated. 
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One object of the invention is to provide a method of quantifying the 
deformation of the organ without suffering the local errors in determining the marking lines. 

In fact, a method in accordance with the introductory paragraph is 
characteristic according to the invention in that it comprises steps of: 
5 • calculating positions of the marked points on at least two images, successive or not, 

• determining parameters of an explicit mathematical expression of the deformation of the 
organ or part of the organ observed between the two images from positions in a set of 
marked points on the two images, said set of marked points containing the marked points 
present on the surface of the organ or at least the marked points present on a part of the 
1 0 surface of the organ. 

Regularization by an explicit mathematical expression for quantifying the 
deformation replaces the approximation by interpolation of the movement at each point, 
which is the method chosen in the document of the state of the art cited. The parameters of 
the expression are obtained from a set of marked points containing the marked points present 
15 on the surface of the organ or at least the marked points present on part of the surface of the 
organ. The mathematical expression obtained is then at least valid at any point on this surface 
or on said part of this surface. This approximation estimates a movement in the very strict 
sense of least squares with an explicit mathematical regularization. This regularization 
corrects the noise. 

20 The invention is applied particularly to the heart, which amongst other things 

exhibits deformation in rotation and contraction. In a particular implementation of the 
invention, the mathematical expression is defined in a polar reference frame. The center of 
the reference frame is defined either automatically by calculating a center of gravity, or 
manually by a user. The center of gravity can then be defined as the center of gravity of the 

25 image or as the center of gravity of a surface defined, for example, by segmentation of the 
image. The position of this center can be approximate without appreciably affecting the 
determination of the mathematical expression. The mathematical expression will 
advantageously be chosen as being able to express deformations close to those expected for 
the organ being imaged. Thus the similarities can take account of the rigid deformation of the 

30 heart. In a preferred implementation of the invention, the mathematical expression is derived 
from the expression of a similarity, that is to say, is of the form, in the centered reference 
frame of center o : 
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This particular expression in fact makes it possible to quantify the true 
deformation of the heart with great fidelity. 

This expression does not however make it possible to take account of the 
5 dependency of the deformation in terms of radius. In a particular embodiment, a corrective 
term which is a function of the radius and the polar angle is introduced into the mathematical 
expression of the deformation, said corrective term including parameters determined a 
posteriori from the determination of the first mathematical expression using a set of marked 
points on the two images. The regularization process then takes place in two steps: 

O 1 0 calculation of the ak values and then radial correction. These two steps are iterated until there 

.""3 

yi is convergence, obtained in general after 2 or 3 iterations. 

~ : The mathematical expression obtained includes a certain number of 

IP parameters representing rigid deformations and elastic deformations of the organ. Knowledge 



of these parameters is important for the detection of abnormalities in the behavior of the 
15 organ by a practitioner. In particular, equipment or apparatus for implementing the method 
11 J according to the invention comprises means of extracting the parameters from the 

lit 

?>\ mathematical expression of the deformation corresponding to rigid deformations and means 



of displaying the change in these parameters during the sequence. 

The practitioner may also wish to know the deformation of a particular 

20 structure of the organ, for example a structure visible on the image in the form of a contour. 
In a particular embodiment, the image processing apparatus comprises means for defining a 
structure per unit length of an image in the sequence, means for applying the mathematical 
expression of the deformation to said structure per unit length and means for displaying the 
deformation undergone by said structure per unit length. 

25 The structure per unit length can be defined automatically or be defined 

manually by the practitioner, on one of the images in the sequence. The deformation of this 
structure can be followed from one image to the following one in the sequence by means of 
the mathematical expression determined according to the invention. The structure per unit 
length can thus follow a contour which is visible on the image and represents a physical 

30 structure of the organ (for example, the epicardium or the endocardium), applying the 

deformation then makes it possible to follow the movements of the physical structure. The 
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structure per unit length may also make it possible to effect a segmentation of the image, and 
said segmentation will then be followed throughout the sequence with the following of the 
deformation of the structure per unit length. Such a segmentation may make it possible to 
define the surface on which the mathematical expression is determined: the quantity of 
calculations is then reduced and the determination of the mathematical expression is more 
precise since it can be carried out on more restricted surfaces. The structure per unit length 
may be any structure included not strictly on the surface on which the mathematical 
expression of the deformation is defined. 

The invention will be further described with reference to examples of 
embodiment shown in the drawings to which, however, the invention is not restricted. 

Fig. 1 depicts a diagram of an image processing method according to the 

invention, 

Fig. 2 presents a CSPAMM image of a heart, said image belonging to a 
sequence of images and being taken after the magnetization pulse, 

Fig. 3 depicts an image processing apparatus according to a particular 
embodiment of the invention, 

Figs. 4a and 4b depict respectively the change over time in the rotation and 
contraction parameters during three image sequences, each being taken at different places on 
the heart, 

Fig. 5 depicts the following of the deformation of a circle to which the given 
expression of the deformation during the sequence is applied, and 

Fig. 6 depicts an apparatus for capturing and processing images according to 

the invention. 

Figure 1 depicts a diagram of an image processing method according to the 
invention. Said method is applicable to images belonging to an image sequence of at least 
two images IM(tl) and IM(t2) taken at two times tl and t2 of an organ or part of an organ 
which is deformable over time. Said organ or said part of the organ being visible on the 
images in the form of a surface called the surface of the organ, said surface including 
characteristic points whose correspondences are determined from one image to another in a 
sequence, said characteristic points are denoted marked points. Said organ or said part of an 
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organ may, for example, be marked by magnetic resonance spatial modulation. Hereinafter, 
the invention is described more particularly in the case of this marking by magnetic 
resonance spatial modulation. 

The technique of marking by magnetic resonance spatial modulation includes 
5 in particular the SPAMM and CSPAMM techniques for obtaining images in which the 
marking is visible on the images along marking lines which may be of different geometries 
when they are generated in the organ (straight lines, curves etc). Said marking lines deform 
whilst following the material deformation of the organ. In the images obtained by means of 
the techniques mentioned above, the lines corresponding to the spatial magnetization minima 
10 are dark lines and can easily be located. 

Magnetic resonance spatial modulation is in general used by taking series of 
images of the organ marked at successive and regular times. These series of images are 
gj referred to as image sequences and the deformation of the organ is observed by means of the 

hi deformation of the marking lines which constitute a kind of frame attached to the organ. Said 

y i 

U1 1 5 frame may have various aspects: parallel straight lines, a grid consisting of straight lines in 
||i two directions etc. A technique known as 'Slice Following' makes it possible to follow the 

deformation of a section of the organ even if the plane of this section moves in a direction 
substantially perpendicular to this plane during the sequence. 

In the case of a periodic deformation of the organ, several sequences of images 
of the same organ taken for similar successive deformations show the same deformation on 
each similar image, that is to say, sampled at the same time within the deformation period. In 
this case, the similar images can be combined so that the frames of the two images are visible 
on the new image resulting from the combination. In this way a new image sequence is 
defined by effecting this combination on all the images in the sequence. This new sequence 
in general contains more information than the original more simple sequences. 

According to the sequence acquisition times, which depend on the marking 
chosen, it may be advantageous either to work on a single image sequence of the organ 
marked with a complex marking or to work on a combination of several image sequences 
(generally two) of the organ marked for each occasion with a simple marking, said 
combination defining a new sequence used next in the image processing method according to 
the invention. 

Figure 2 presents an image of a heart, said image belonging to an image 
sequence and being taken approximately 9 ms after the magnetization pulse. Two sets of 
parallel lines corresponding to light intensity minima are observed, the parallel lines in one 
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set being perpendicular to the parallel lines in the other set. The image sequence from which 
this example is taken thus has marking lines in two distinct directions and may thus be the 
result either of a direct acquisition of an image sequence of the organ marked in both 
directions or the combination of two acquisitions of sequences, each of the two sequences 
5 being marked in one of the two directions. 

Two types of marking lines may be used in a method according to the 
invention. In Figure 2, the marking lines corresponding to intensity minima, that is to say, to 
magnetic resonance minima, can easily be located. The marking lines corresponding to 
intensity maxima and corresponding to magnetic resonance maxima are, however, also 
1 0 detectable, even if they are less easy to detect. For example, by derivation from the image 
intensity profile it is possible to locate the lines corresponding to the magnetization maxima. 
The use of these two types of marking lines increases the information on the image since the 
Q marking frame is closer together: the number of intersection points between marking lines 

|p and therefore the number of marked points is higher. 

Ill 1 5 With regard to the quality of the photographs of a sequence, the CSPAMM 

||1 technique makes it possible notably to obtain a persistent contrast on a sequence. This is in 

„ particular useful in the case where the marking lines corresponding to the magnetic resonance 

d maxima are used, the persistent contrast helping with the localization of the intensity minima. 

m The image processing method according to the invention processes images 

S 20 where marked points for which it is possible to establish correspondences from one image to 
pi the other are present. In the case of magnetic resonance spatial modulation, the marking lines 

are such that there are points of intersection between several marking lines. It is easy to 
establish matches from one image to another for these points of intersection, which are 
hereinafter referred to as marked points MP. They may be points of intersection between 
25 marking lines of any form and may be directly visible on the marking frame or be visible 
only after a combination of several sequences, giving a new image sequence. 

The method according to the invention (Figure 1) includes a first step CALC 
of calculating the positions of the marked points. This may, in the example of magnetic 
resonance spatial modulation, be effected by the use of the method described in patent 
30 application PHF0001 16 included herein by reference. In this patent application, points which 
are candidates for belonging to a given marking line are detected before means of predicting 
the movement of the marking line are used to identify the marking line and the points 
belonging to it and before an equation for the line is calculated. Whilst the equations for the 
marking lines have been determined, calculating the positions of the marked points is easy. 
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The marked points MP(tl) and MP(t2) are the points for which a correspondence is 
established from one image IM(tl) to the image IM(t2), the two times tl and t2 being able to 
be successive or not in the image sequence and tl being able to be either subsequent to or 
prior to t2. 

5 The method according to the invention next includes a step DET of 

determining an explicit mathematical expression f of the deformation of the organ or of the 
part of the organ observed between the image IM(tl) and IM(t2) from a set MP' of marked 
points whose positions are defined by MP'(tl) on the image IM(tl) and MP'(t2) on the image 
IM(t2). Said sets MP' are included not strictly in the sets MP of marked points and include 
1 0 the marked points present on the surface of the organ or at least those present on part of this 
surface. The parameters of the mathematical expression are generally determined by least 
squares approximation from the positions of the marked points whose positions are known on 
Q the two images and therefore whose movement is known between tl and t2. The 

{13 mathematical expression may, for example, be a similarity which takes account of the rigid 

f] 15 deformations. 

• " In a particular embodiment, the mathematical expression of the deformation is 

defined in the complex plane. The deformation can be defined in a polar reference frame. In 
P the particular case, in which the organ is a heart, it is easy to define a center positioned 

hi approximately at the center of gravity of the surface of the organ as seen on the image. 

j*j 20 In the preferred embodiment of the invention, the mathematical expression is 

IIJ derived from a similarity which can be written in a form relating to a point o chosen in any 

manner, but generally chosen as being approximately the center of gravity of the surface 
observed: 

f(z) = | z-o | (ae w )+ d,6= arg(z-o). 
25 The expression of the similarity is modified by introducing a Fourier series 

into the expression in order to take account of deformations which are more elastic than a 
simple similarity: 



+ d , 0 = arg(z-o), 



This expression takes account of a global semi-elastic deformation in a 
centered reference frame. It is defined by 2x(2N+l)+2 real parameters which are defined 
from marked points of the set MP'. These values are in general overevaluated since there are 
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more marked points in MP' than parameters. This is in particular the case when the marking 
lines corresponding to the magnetic resonance maxima are used, the number of points of 
intersection between the lines then being high. The overevaluation of the parameters makes it 
possible to smooth the noise. 
5 In the case of the heart, the endocardium is notably more contractile than the 

epicardium and consequently the more the center of the myocardium is approached the 
greater the magnitude of the radial movement. A corrective term which is a function of the 
radius and the polar angle is advantageously added a posteriori to the determination of the 
explicit mathematical expression f in order to add a dependency in terms of radius. This 
10 corrective term is also determined using the positions of the marked points MP' on the two 
images but this determination is effected after the determination of the parameters a k of the 
deformation f(z). 

p In an advantageous implementation, the corrective term is defined by angular 



sectors s of the image and is of the form: 



111 ( TL io-o k ) 

The term y s (r) is a polynomial in terms of r independent of 0 defined on the 
M> angular sector s of the image according to the positions of the marked points on the angular 

| I1 sector s on the two images. The Lagrange polynomial interpolator is then used to take 



account of the dependency in terms of 0, B s being the center angle of the angular sector s. 

fii 

20 If the corrective term were determined at the same time as the function f, there 

would exist a multiplicity of writing of the set except if the function f were constrained. 
However, f gives a mathematical expression of the global deformation. The corrective term 
represents the physiological behavior rather than the kinematic deformation. 

Overall, the elasticity of the mathematical expression is controlled by the 

25 choice of N, the Fourier order (N=3 is generally sufficient) and, where the corrective term is 
introduced, by the number of angular sectors considered for the approximation of the 
corrective term and the choice of the degree of the polynomial y s . 

Figure 3 depicts image processing equipment according to a particular 
embodiment of the invention. This equipment is in relationship with means ACQ of acquiring 

30 sequences SIM of X images. This equipment includes the means CALC of calculating the 
positions of the marked points on two images, successive or not, and means DET of 
determining parameters of an explicit mathematical expression of the deformation of the 
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organ or of the part of the organ observed between the two images from positions of a set of 
marked points on the two images, the said set of marked points containing the marked points 
present on the surface of the organ and at least the marked points present on part of this 
surface. 

5 In the particular embodiment depicted in Figure 3, the method described in the 

Figure 1 for two images, successive or not, is iterated on the set of successive images IM(ti) 
of the image sequence SIM. After an initialization for a counter initialized to i=0, where the 
positions of the marked points MP(0) are calculated and stored in the memory MEM, the 
process described below is initiated with i=l . This initialization is not explicitly depicted in 
1 0 the figure since it is the particular case of the general scheme where i=0, where f is the null 
function and where MP(ti-i)=MP(0). 

An image IM(tj) is extracted from the sequences of images SIM. The positions 
jp of the marked points MP(t;) are calculated by calculation means CALC. These positions are 

ijj stored in a memory MEM and are supplied to means DET of determining a mathematical 

1 5 expression of the deformation. The positions of a set of marked points MP'(ti_i) of the 
01 previous image IM(tj-i) are extracted from the memory MEM and supplied to the means DET 

for determining the expression of the deformation f(tu; ti). In the embodiment depicted in 
P Figure 3, the mathematical expression of the deformation is then stored in the memory MEM 

flj and the counter is incremented to i=i+l . 

Iff 20 The iteration of the determination of the deformation on a sequence SIM 

fU makes it possible to evaluate the parameters of the deformation and their change over time. 

The mathematical expression of the deformation includes amongst other things the rigid 
contraction and rotation parameters which are included in the complex parameter ai. 
Knowledge of these parameters and of the changes in them makes it possible to extract them 
25 from the expression and to trace them as a function of time. Figures 4a and 4b depict 

respectively, in two graphs, the change over time as a function of i of the rotation ROT and of 
the contraction CONT extracted from ai for two image sequences IM(i), each being taken for 
two different points on the heart: the base (curve 1) and the apex (curve 2). The extraction of 
these parameters and the display thereof require knowledge within the capability of experts. 
30 These graphs are particularly useful for the practitioner, who can thus visualize the overall 

deformation of the organ during the sequence. In the example of the heart, the practitioner, by 
means of this tool, visualizes the overall rigid deformation of the heart and can detect 
abnormalities therein. 
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In accordance with Figure 5, it is also possible to visualize the global 
deformation with its rigid and elastic components by applying a circle to the part of the 
mathematical expression independent of the radius. In the preferred embodiment, the 
mathematical expression applied is of the form: 



f(z) = \z-o 



a k e 



k= 

. Jfc*0 



vd , e = arg(z-o), (a k )eC 2N+1 



f 0 {e) 

The visualization of the deformation of the circle is, for the practitioner, a 
powerful and user-friendly tool for detecting abnormalities of the cardiac deformation. An 
example of this visualization of the deformation is presented in Figures 5a and b. 

Figure 6 depicts an image acquisition apparatus APP, the said apparatus 
comprising means ACQ of acquiring sequences of at least two images of an organ or of a part 
of an organ caused to deform over time, the said organ or the said part of the organ being 
visible in the images in the form of a surface referred to as the surface of the organ, the said 
surface including characteristic points whose correspondences are determined from one 
image to another in the sequence, means REP of visual representation of these images, which 
can comprise a video mode to follow the deformation during the sequence, and an image 
processing apparatus DEV as described previously. 



